HTML Tag Based Metrics for use in Web Page Type Classification

Traditional machine learning classifications of HTML documents focus on features drawn from terms in the documents, the link structure of groups of documents, or a combination of both. These techniques attempt to generate topical classifications of documents, with the hopes of mirroring a human's classification of pages into subject areas, thus facilitating retrieval. This paper presents an alternative method that aims at generating a "type-wise" classification of HTML documents. The types explored in this paper include tables, indexes, tables of contents, and textual content pages. These types of pages are of particular significance to the classification of documents on statistical web sites, which is one goal of the GovStat Project (http://www.ils.unc.edu/govstat), but also hold significance to HTML document collections at large.